Selection bias

Selection bias is a statistical bias in which there is an error in choosing the individuals or groups to take part in a scientific study.[1] It is sometimes referred to as the selection effect. The term "selection bias" most often refers to the distortion of a statistical analysis, resulting from the method of collecting samples. If the selection bias is not taken into account then certain conclusions drawn may be wrong.

Contents

Types

There are many types of possible selection bias, including:

Sampling bias

Sampling bias is systematic error due to a non-random sample of a population,[2] causing some members of the population to be less likely to be included than others, resulting in a biased sample, defined as a statistical sample of a population (or non-human factors) in which all participants are not equally balanced or objectively represented.[3] It is mostly classified as a subtype of selection bias,[4] sometimes specifically termed sample selection bias,[5][6] but some classify it as a separate type of bias.[7]

A distinction, albeit not universally accepted, of sampling bias is that it undermines the external validity of a test (the ability of its results to be generalized to the rest of the population), while selection bias mainly addresses internal validity for differences or similarities found in the sample at hand. In this sense, errors occurring in the process of gathering the sample or cohort cause sampling bias, while errors in any process thereafter cause selection bias.

Examples of sampling bias include self-selection, pre-screening of trial participants, discounting trial subjects/tests that did not run to completion and migration bias by excluding subjects who have recently moved into or out of the study area.

Time interval

Exposure

Data

Studies

Attrition

Attrition bias is a kind of selection bias caused by attrition (loss of participants),[12] discounting trial subjects/tests that did not run to completion. It includes dropout, nonresponse (lower response rate), withdrawal and protocol deviators. It gives biased results where it is unequal in regard to exposure and/or outcome. For example, in a test of a dieting program, the researcher may simply reject everyone who drops out of the trial, but most of those who drop out are those for whom it was not working. Different loss of subjects in intervention and comparison group may change the characteristics of these groups and outcomes irrespective of the studied intervention.[12]

Observer selection

Data is filtered not only by study design and measurement, but by the necessary precondition that there has to be someone doing a study. In situations where the existence of the observer or the study is correlated with the data observation selection effects occur, and anthropic reasoning is required.[13]

An example is the past impact event record of Earth: if large impacts cause mass extinctions and ecological disruptions precluding the evolution of intelligent observers for long periods, humans will not observe any evidence of large impacts in the recent past (since they would have prevented humans from evolving). Hence there is a potential bias in the impact record of Earth.[14] Astronomical existential risks might similarly be underestimated due to selection bias, and an anthropic correction has to be introduced.[15]

Avoidance

In the general case, selection biases cannot be overcome with statistical analysis of existing data alone, though Heckman correction may be used in special cases. An informal assessment of the degree of selection bias can be made by examining correlations between exogenous (background) variables and a treatment indicator. However, in regression models, it is correlation between unobserved determinants of the outcome and unobserved determinants of selection into the sample which bias estimates, and this correlation between unobservables cannot be directly assessed by the observed determinants of treatment.[16]

Related issues

Selection bias is closely related to:

See also

Notes

  1. ^ Dictionary of Cancer Terms → selection bias Retrieved on September 23, 2009.
  2. ^ Medical Dictionary - 'Sampling Bias' Retrieved on September 23, 2009
  3. ^ TheFreeDictionary → biased sample Retrieved on 2009-09-23. Site in turn cites: Mosby's Medical Dictionary, 8th edition.
  4. ^ Dictionary of Cancer Terms → Selection Bias Retrieved on September 23, 2009
  5. ^ The effects of sample selection bias on racial differences in child abuse reporting Ards S, Chung C, Myers SL Jr. Child Abuse Negl. 1999 Dec;23(12):1209; author reply 1211-5. PMID 9504213
  6. ^ Sample Selection Bias Correction Theory Corinna Cortes, Mehryar Mohri, Michael Riley, and Afshin Rostamizadeh. New York University.
  7. ^ Page 262 in: Behavioral Science. Board Review Series. By Barbara Fadem. ISBN 0781782570, 9780781782579. 216 pages
  8. ^ a b Feinstein AR, Horwitz RI (November 1978). "A critique of the statistical evidence associating estrogens with endometrial cancer". Cancer Res. 38 (11 Pt 2): 4001–5. PMID 698947. 
  9. ^ Tamim H, Monfared AA, LeLorier J (March 2007). "Application of lag-time into exposure definitions to control for protopathic bias". Pharmacoepidemiol Drug Saf 16 (3): 250–8. doi:10.1002/pds.1360. PMID 17245804. 
  10. ^ Matthew R. Weir (2005). Hypertension (Key Diseases) (Acp Key Diseases Series). Philadelphia, Pa: American College of Physicians. p. 159. ISBN 1-930513-58-5. 
  11. ^ Kruskal, W. (1960) Some notes on wild observations, Technometrics.
  12. ^ a b Jüni P, Egger M. Empirical evidence of attrition bias in clinical trials. Int J Epidemiol. 2005 Feb;34(1):87-8.
  13. ^ Nick Bostrom, Anthropic Bias: Observation selection effects in science and philosophy. Routledge, New York 2002
  14. ^ Milan M. Církovic, Anders Sandberg, and Nick Bostrom. Anthropic Shadow: Observation Selection Effects and Human Extinction Risks. Risk Analysis, Vol. 30, No. 10, 2010.
  15. ^ Max Tegmark and Nick Bostrom, How unlikely is a doomsday catastrophe? Nature, Vol. 438 (2005): 75. arXiv:astro-ph/0512204
  16. ^ Heckman, J. (1979) Sample selection bias as a specification error. Econometrica, 47, 153–61.